Tracking system for head-mounted display apparatus and method of tracking

ABSTRACT

A tracking system for use in head-mounted display apparatus includes at least one emitter that emits signals; a first receiver and a second receiver that sense the emitted signals and generate sensor data, the first receiver and the second receiver being arranged on a first portion and a second portion, respectively, wherein the first portion faces a user when the head-mounted display apparatus is worn by the user, and the second portion is a part of a user-interaction controller of a head-mounted display; and a processor configured to process the generated sensor data to determine relative positions and orientations of first receiver and second receiver with respect to the emitter, and to determine a relative position and orientation of the second receiver with respect to the first receiver.

TECHNICAL FIELD

The present disclosure relates generally to tracking systems; and morespecifically, to tracking systems for use in head-mounted displayapparatuses comprising emitters, receivers and processors. Furthermore,the present disclosure also relates to methods of tracking.

BACKGROUND

Nowadays, several specialized devices (for example, such as VirtualReality devices, Augmented Reality (AR) devices, Mixed Reality (MR)devices, and the like) are being used by users to experience andinteract with simulated environments (for example, such as AR, MR andthe like). Such simulated environments use contemporary techniques (forexample, such as stereoscopy) to provide the user with a feeling ofimmersion within the simulated environments.

Generally, such specialized devices include interaction controllers (forexample, such as remote controllers, joysticks, and the like) that areused by the users to interact with the simulated environment. Whileusing the specialized devices, the users often move said interactioncontrollers within a physical environment whereat the users are present.In order to properly emulate said physical-world movement of theinteraction controllers in the simulated environment, the specializeddevices require knowledge of a position and an orientation of theinteraction controllers in the physical environment.

Presently, the specialized devices employ tracking equipment to trackthe position and the orientation of the interaction controllers withinthe physical environment. Examples of such tracking equipment includeexternal cameras that illuminate the physical environment with a lightpattern, laser beacons and diode firing-based equipment employingtriangulation, electromagnetic field generators, and the like. As anexample, the electromagnetic field generators include a transmitter thatemits electromagnetic signals, and receivers that receive the emittedelectromagnetic signals. Relative positions and orientations of thetransmitter and the receivers with respect to each other are determinedusing the received signals.

However, such tracking equipment have several limitations associatedtherewith. In an example, the transmitter used in the electromagneticfield generators is large in size, power consuming and is bulky. As aresult, arranging the transmitter within the specialized devices withoutadversely impacting usability of the specialized devices is difficult.In another example, when multiple interaction controllers are used in asame physical environment, said interaction controllers interfere witheach other. This considerably reduces accuracy and efficiency of thetracking equipment.

Therefore, in light of the foregoing discussion, there exists a need toovercome the aforementioned drawbacks associated with conventionaltracking equipment for specialized devices.

SUMMARY

The present disclosure seeks to provide a tracking system for use in ahead-mounted display apparatus. The present disclosure also seeks toprovide a method of tracking. The present disclosure seeks to provide asolution to the existing problems of sub-optimal design and inaccuraciesamong existing tracking equipment for specialized devices. An aim of thepresent disclosure is to provide a solution that overcomes at leastpartially the problems encountered in prior art, and provides awell-designed, simple, accurate and user-friendly tracking system forthe head-mounted display apparatus.

In one aspect, an embodiment of the present disclosure provides atracking system for use in a head-mounted display apparatus, thetracking system comprising:

-   -   at least one emitter that, in operation, emits signals;    -   a first receiver and a second receiver that, in operation, sense        the emitted signals and generate sensor data, the first receiver        and the second receiver being arranged on a first portion and a        second portion of the head-mounted display apparatus,        respectively, wherein the first portion faces a user when the        head-mounted display apparatus is worn by the user, and the        second portion is a part of a user-interaction controller of the        head-mounted display apparatus; and    -   a processor configured to process the generated sensor data to        determine relative positions and orientations of the first        receiver and the second receiver with respect to the at least        one emitter, and to determine, based on the determined relative        positions and orientations, a relative position and orientation        of the second receiver with respect to the first receiver.

In another aspect, an embodiment of the present disclosure provides amethod of tracking, the method comprising:

-   -   emitting signals, via at least one emitter;    -   sensing, via a first receiver and a second receiver, the emitted        signals and generating sensor data, the first receiver and the        second receiver being arranged on a first portion and a second        portion of a head-mounted display apparatus, respectively,        wherein the first portion faces a user when the head-mounted        display apparatus is worn by the user, and the second portion is        a part of a user-interaction controller of the head-mounted        display apparatus;    -   processing the generated sensor data to determine relative        positions and orientations of the first receiver and the second        receiver with respect to the at least one emitter; and    -   determining, based on the determined relative positions and        orientations, a relative position and orientation of the second        receiver with respect to the first receiver.

Embodiments of the present disclosure substantially eliminate or atleast partially address the aforementioned problems in the prior art,and enable accurate tracking of a pose of the head-mounted displayapparatus, via the tracking system.

Additional aspects, advantages, features and objects of the presentdisclosure would be made apparent from the drawings and the detaileddescription of the illustrative embodiments construed in conjunctionwith the appended claims that follow.

It will be appreciated that features of the present disclosure aresusceptible to being combined in various combinations without departingfrom the scope of the present disclosure as defined by the appendedclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

The summary above, as well as the following detailed description ofillustrative embodiments, is better understood when read in conjunctionwith the appended drawings. For the purpose of illustrating the presentdisclosure, exemplary constructions of the disclosure are shown in thedrawings. However, the present disclosure is not limited to specificmethods and instrumentalities disclosed herein. Moreover, those skilledin the art will understand that the drawings are not to scale. Whereverpossible, like elements have been indicated by identical numbers.

Embodiments of the present disclosure will now be described, by way ofexample only, with reference to the following diagrams wherein:

FIGS. 1 and 2 illustrate block diagrams of architectures of a trackingsystem for use in a head-mounted display apparatus, in accordance withdifferent embodiments of the present disclosure;

FIGS. 3 and 4 are schematic illustrations of a tracking system in use,in accordance with different embodiments of the present disclosure; and

FIG. 5 illustrates steps of a method of tracking, in accordance with anembodiment of the present disclosure.

In the accompanying drawings, an underlined number is employed torepresent an item over which the underlined number is positioned or anitem to which the underlined number is adjacent. A non-underlined numberrelates to an item identified by a line linking the non-underlinednumber to the item. When a number is non-underlined and accompanied byan associated arrow, the non-underlined number is used to identify ageneral item at which the arrow is pointing.

DETAILED DESCRIPTION OF EMBODIMENTS

The following detailed description illustrates embodiments of thepresent disclosure and ways in which they can be implemented. Althoughsome modes of carrying out the present disclosure have been disclosed,those skilled in the art would recognize that other embodiments forcarrying out or practising the present disclosure are also possible.

In one aspect, an embodiment of the present disclosure provides atracking system for use in a head-mounted display apparatus, thetracking system comprising:

-   -   at least one emitter that, in operation, emits signals;    -   a first receiver and a second receiver that, in operation, sense        the emitted signals and generate sensor data, the first receiver        and the second receiver being arranged on a first portion and a        second portion of the head-mounted display apparatus,        respectively, wherein the first portion faces a user when the        head-mounted display apparatus is worn by the user, and the        second portion is a part of    -   a user-interaction controller of the head-mounted display        apparatus; and a processor configured to process the generated        sensor data to determine relative positions and orientations of        the first receiver and the second receiver with respect to the        at least one emitter, and to determine, based on the determined        relative positions and orientations, a relative position and        orientation of the second receiver with respect to the first        receiver.

In another aspect, an embodiment of the present disclosure provides amethod of tracking, the method comprising:

-   -   emitting signals, via at least one emitter;    -   sensing, via a first receiver and a second receiver, the emitted        signals and generating sensor data, the first receiver and the        second receiver being arranged on a first portion and a second        portion of a head-mounted display apparatus, respectively,        wherein the first portion faces a user when the head-mounted        display apparatus is worn by the user, and the second portion is        a part of a user-interaction controller of the head-mounted        display apparatus;    -   processing the generated sensor data to determine relative        positions and orientations of the first receiver and the second        receiver with respect to the at least one emitter; and    -   determining, based on the determined relative positions and        orientations, a relative position and orientation of the second        receiver with respect to the first receiver.

The present disclosure provides the aforementioned tracking system andthe aforementioned method of tracking. In the tracking system, anarrangement of the at least one emitter and the receivers is optimallyselected to facilitate accurate tracking of the head-mounted displayapparatus without adversely impacting user-friendliness of thehead-mounted display apparatus. In other words, components of saidtracking system can be compactly arranged on the head-mounted displayapparatus, thereby enabling easy movement of the head-mounted displayapparatus by the user. The aforesaid method of tracking is simple, fast,and yields reliable results.

Throughout the present disclosure, the term “head-mounted displayapparatus” refers to specialized equipment that is configured to presenta simulated environment to a user when said equipment in operation isworn by the user on his/her head. In such an instance, the head-mounteddisplay apparatus acts as a device (for example, such as a VirtualReality (VR) headset, a pair of VR glasses, an Augmented Reality (AR)headset, a pair of AR glasses, a Mixed Reality (MR) headset, a pair ofMR glasses, and the like) that is operable to present a visual scene ofthe simulated environment to the user. In an example, the visual scenemay be a virtual reality movie. In another example, the visual scene maybe an educational augmented reality tutorial. In yet another example,the visual scene may be a mixed reality game.

Throughout the present disclosure, the term “tracking system” refers tospecialized equipment for detecting and following a pose of thehead-mounted display apparatus. Herein, the term “pose” encompasses bothposition and orientation.

The tracking system is a true six Degrees of Freedom (6DoF) trackingsystem. Notably, the tracking system allows for tracking the positionand the orientation of the head-mounted display apparatus in threedimensions. In particular, the tracking system is configured to tracktranslational movements (namely, surge, heave and sway movements) androtational movements (namely, roll, pitch and yaw movements) of thehead-mounted display apparatus within a three-dimensional space of areal-world environment. It will be appreciated that use of said trackingsystem in the head-mounted display apparatus allows for providing atruly immersive simulated reality experience to the user. Beneficially,the visual scene to be presented to the user can be adjusted accordingto a current pose of the head-mounted display apparatus for providing arealistic perception of the simulated environment to the user.

Throughout the present disclosure, the term “emitter” refers tospecialized equipment that, in operation, emits signals. The emittedsignals disperse within the real-world environment, and can be sensed byreceiver(s) arranged within said real-world environment. The at leastone emitter can be understood to be at least one “signal source” or atleast one “signal transmitter”.

As an example, a given emitter may be an orthogonal coil emittercomprising three coils, said coils being arranged orthogonally to oneanother. When such an emitter would be driven, for example, by analternating electric current (continuous, pulsed, or a combination ofboth), the three coils would emit orthogonal signals that spread acrossthe three-dimensional space of the real-world environment. Moreover,signals employed to drive the three coils may have different attributes.For example, three signals employed to drive the three coils may betime-separable or frequency-separable.

Optionally, the at least one emitter is an isotropic emitter. In otherwords, the at least one emitter emits a same intensity of the signals inall directions.

Optionally, the at least one emitter emits signals within a predefinedregion of the real-world environment. As an example, the at least oneemitter may emit signals within a 100 cubic meters region of thereal-world environment.

In an embodiment, the emitted signals are electromagnetic signals. Suchelectromagnetic signals can be understood to be electromagnetic waves ofan electromagnetic field. The electromagnetic field is a combination ofan electric field and a magnetic field. Since an intensity of theelectromagnetic field varies spatially within the real-worldenvironment, such electromagnetic signals can be employed to accuratelytrack the pose of the head-mounted display apparatus within thereal-world environment. An intensity of the electromagnetic field sensedby a given receiver reduces with increase in a distance between thegiven receiver and the at least one emitter, thereby allowing fortracking the translational movements of the head-mounted displayapparatus. Moreover, a distribution of the intensity of theelectromagnetic field along three coordinate axes of the given receiverchanges upon rotation of the given receiver, thereby allowing fortracking the rotational movements of the head-mounted display apparatus.

It will be appreciated that the term “electromagnetic signals”encompasses signals having frequencies lying within the electromagneticspectrum. Notably, the electromagnetic signals include low-frequencyradio signals, high-frequency radio signals, visible signals (light),infrared signals, and the like. Moreover, tracking the pose of thehead-mounted display apparatus using the magnetic field of suchelectromagnetic signals may not require that a line-of-sight be presentbetween the at least one emitter and the first and second receivers.Therefore, such electromagnetic signals can be effectively employed forpose tracking even when there exist physical obstruction(s) between theat least one emitter and the first and second receivers.

As an example, the emitted signals are radio signals. Such radio signalscan be understood to be radio waves. Optionally, the radio signals havea frequency lying in the Industrial, Scientific and Medical (ISM) radiobands. In such a case, the tracking system can be used without obtaininga license for providing short range tracking within the real-worldenvironment.

In another embodiment, the emitted signals are acoustic signals.Optionally, the acoustic signals are ultrasound signals. Notably, timetaken by the acoustic signals to travel from the at least one emitter toa given receiver is utilized for tracking the pose of the head-mounteddisplay apparatus. In particular, the time taken by the acoustic signalsto travel from the at least one emitter to the given receiver increaseswith an increase in a distance between the at least one emitter and thegiven receiver. This allows for tracking the translational movements ofthe head-mounted display apparatus using the acoustic signals. Moreover,when an arrangement of the first and second receivers on thehead-mounted display apparatus is known, a time difference between thetimes taken by the acoustic signals to travel from the at least oneemitter to both the first and second receivers allows for tracking therotational movements of the head-mounted display apparatus.

Throughout the present disclosure, the term “receiver” refers tospecialized equipment that, in operation, senses the emitted signals.Upon sensing the emitted signals, a given receiver generates sensordata. Herein, the terms “first receiver” and “second receiver” do notdenote any order, quantity, or importance of receivers, but rather areused to distinguish one receiver from another.

It will be appreciated that the first and second receivers arecompatible with the at least one emitter. In other words, the first andsecond receivers are capable of sensing a type of the signals that areemitted by the at least one emitter. Notably, emitter(s) and receiversemployed for different tracking technologies (for example, such asradio-frequency based tracking, visible-light based tracking,acoustics-based tracking, and the like) would be different. As anexample, when the emitted signals are electromagnetic signals havingradio frequency, the emitter and the first and second receivers includecompatible radio frequency antennas, and additionally, amplifiers,signal processing circuitry and the like. In such an example, the firstand second receivers may be implemented as electromagnetic coilsconfigured to sense the electromagnetic signals having radio frequency.As another example, when the emitted signals are acoustic signals, thefirst and second receivers are implemented as acoustic receivers.

Furthermore, it will be appreciated that a given emitter-receiver pairis tuned to a same signal frequency or wavelength for implementing theaforesaid sensing operation. Optionally, the tracking system comprises afrequency filter configured to separate frequencies employed fordifferent emitter-receiver pairs from one another. This allows forclearly distinguishing emitted signals corresponding to oneemitter-receiver pair from emitted signals corresponding to anotheremitter-receiver pair, thereby, minimizing tracking errors within thetracking system.

Moreover, it will be appreciated that the first and second receivers arelightweight and small in size. Therefore, a plurality of receivers canbeneficially be placed in the first and second portions, and optionally,in other portions of the head-mounted display apparatus to increase anaccuracy of the tracking system.

Optionally, the generated sensor data comprises measurements of at leastone signal characteristic of the received signals. Optionally, in thisregard, the at least one signal characteristic comprises at least oneof: a strength, a phase, a wavelength, a frequency, a time-of-flight, anangle of arrival, a polarization, a Doppler spread, a delay spread, adelay signature.

Optionally, the at least one signal characteristic to be measureddepends upon the type, and optionally, the wavelength of the emittedsignals. For example, when the wavelength of the emitted signals is ofthe order of few kilometres, the phase of said signals may not bemeasured. However, when the wavelength of the emitted signals is of theorder of few centimetres, the phase of said signals may be measured.

In an example, when the emitted signals are the electromagnetic signals,the generated sensor data may comprise measured strength of inducedcurrent and/or induced voltage in the first and second receivers. When agiven receiver moves within a spatially variable magnetic field of theelectromagnetic signals, a voltage is induced in the given receiver.Said voltage is proportional to a cross product of cross-sectional areaof the given receiver and an intensity of the magnetic field sensed bythe given receiver. Furthermore, due to the aforesaid induced voltage, acurrent is induced in the given receiver. The measured strength of theinduced current and/or the induced voltage may be utilized by theprocessor to determine the pose of the given receiver with respect tothe at least one emitter.

In another example, when the emitted signals are the electromagneticsignals, the generated sensor data may comprise measured values ofstrength (namely, power) of received electromagnetic signals at thefirst and second receivers. Notably, a strength of the emittedelectromagnetic signals is greater than the strengths of the receivedelectromagnetic signals since the emitted electromagnetic signalsundergo attenuation while propagating through the real-worldenvironment. The measured values of the strengths of receivedelectromagnetic signals at the first and second receivers may beutilized by the processor to determine the pose of the first and secondreceivers with respect to the emitter. For electromagnetic signalshaving a frequency within radio spectrum, the aforesaid sensor data isutilized for “radiolocation” of the first and second receivers.

In yet another example, when the emitted signals are the acousticsignals, the generated sensor data comprises time-of-flight measurementsof the acoustic signals. Such time-of-flight measurements are values oftimes taken by the acoustic signals to travel from the at least oneemitter to the first and second receivers. Given a time at which theacoustic signals are emitted by the at least one emitter and times atwhich the acoustic signals are received at the first and secondreceivers, time durations taken by the acoustic signals to travel fromthe at least one emitter to the first and second receivers can bedetermined. Such time durations taken by the acoustic signals to travelfrom the at least one emitter to the first and second receivers may beutilized by the processor to determine the pose of the first and secondreceivers with respect to the at least one emitter.

The first receiver and the second receiver are arranged on the firstportion and the second portion of the head-mounted display apparatus,respectively. It will be appreciated that the first and second portionsof the display apparatus could include additional receiver(s) arrangedthereon. As an example, a third receiver and a fourth receiver may alsobe arranged on the second portion of the display apparatus.

Herein, the terms “first portion” and “second portion” do not denote anyorder, quantity, or importance of portions of the head-mounted displayapparatus, but rather are used to distinguish one portion of thehead-mounted display apparatus from another.

The first portion faces the user when the head-mounted display apparatusis worn by the user. In particular, the first portion is in proximity ofthe user's eyes.

The second portion is a part of the user-interaction controller of thehead-mounted display apparatus. Throughout the present disclosure, theterm “user-interaction controller” refers to specialized equipment thatis used by the user to interact with the simulated environment. Notably,the user interacts with the simulated environment by moving theuser-interaction controller within the real-world environment. Examplesof the user-interaction controller include, but are not limited to aremote controller, a mouse controller, a joystick controller, and asmartphone controller. Additionally, optionally, the user interacts withthe simulated environment by providing an interaction input via theuser-interaction controller. Optionally, in this regard, the userprovides the interaction input using at least one of: a button of theuser-interaction controller, a touch-sensitive display of theuser-interaction controller, a microphone of the user-interactioncontroller. As an example, the user may move the user-interactioncontroller to point towards an object in the simulated environment, andmay press a button on the user-interaction controller to select saidobject.

According to an embodiment, the user-interaction controller is to behand-held by the user. According to another embodiment, theuser-interaction controller is to be worn by the user. In such a case,the user could wear the user-interaction controller on his/her hand,arm, wrist, leg, neck, and the like.

It will be appreciated that arranging the second receiver, andoptionally, the additional receiver(s) on the second portion of thehead-mounted display apparatus allows for accurately tracking the poseof the user-interaction controller. Moreover, since a given receiver islightweight and small in size, arranging the second receiver, andoptionally, the additional receiver(s) on the second portion would notmake the user-interaction controller bulky. The user interactioncontroller can therefore be designed to be lightweight and compact.

The processor could be implemented as hardware, software, firmware or acombination of these. The processor is coupled to various components ofthe tracking system, and is configured to control the operation of thetracking system.

The processor is configured to process the generated sensor data todetermine relative positions and orientations of the first receiver andthe second receiver with respect to the at least one emitter. Notably,the generated sensor data is processed based upon the type of theemitted signals, wherein the processor employs at least one mathematicalformula for said processing. In particular, the at least onemathematical formula to be employed for processing sensor dataassociated with electromagnetic signals is different from the at leastone mathematical formula to be employed for processing sensor dataassociated with acoustic signals.

As an example, when the emitted signals are electromagnetic signals, themeasured strength of induced current and/or induced voltage along threecoordinate axes of the first and second receivers can be used todetermine intensities of the magnetic field (of the electromagneticfield) along the three coordinate axes of the first and secondreceivers. The intensities of the magnetic field along the threecoordinate axes of the first and second receivers may be used todetermine the relative positions and orientations of the first receiverand the second receiver with respect to the at least one emitter.

Moreover, the processor is configured to determine, based on thedetermined relative positions and orientations, a relative position andorientation of the second receiver with respect to the first receiver.Optionally, in this regard, the processor employs at least onemathematical formula pertaining to three-dimensional coordinate geometryfor determining the relative position and orientation of the secondreceiver with respect to the first receiver.

Optionally, the first portion is in a proximity of an image renderer ofthe head-mounted display apparatus, a relative position and orientationof the first receiver with respect to the image renderer being known,wherein the processor is configured to determine a relative position andorientation of the user-interaction controller with respect to a fieldof view that is visible to the user via the image renderer, based on therelative position and orientation of the first receiver with respect tothe image renderer and the relative position and orientation of thesecond receiver with respect to the first receiver.

In an embodiment, the first receiver is directly attached to the imagerenderer. In another embodiment, the first receiver is attached to theimage renderer, via a supporting element.

Optionally, the relative position and orientation of the first receiverwith respect to the image renderer is constant, and is set whilemanufacturing the head-mounted display apparatus. As a result, therelative position and orientation of the first receiver with respect tothe image renderer is known. This also simplifies an internal design ofthe head-mounted display apparatus.

Moreover, optionally, the processor employs at least one mathematicalformula pertaining to three-dimensional coordinate geometry fordetermining the relative position and orientation of theuser-interaction controller with respect to the field of view that isvisible to the user via the image renderer. In such a case, the at leastone mathematical formula is a function of the relative position andorientation of the first receiver with respect to the image renderer,the relative position and orientation of the second receiver withrespect to the first receiver, and the relative position and orientationof the first receiver with respect to the image renderer.

It will be appreciated that arranging the first receiver in proximity ofthe image renderer accommodates for possible alignment issues of thetracking system that may be caused during use of the head-mounteddisplay apparatus. Moreover, magnetic effects of a local magnetic fieldbetween the first receiver (that is arranged in proximity to the imagerenderer) and the at least one emitter can be employed to understandmagnetic effects of an magnetic field between the second receiver (thatis arranged on the user-interface controller) and the at least oneemitter. This allows for improving accuracy and reliability of thetracking system.

It will also be appreciated that determining the relative position andorientation of the user-interaction controller with respect to saidfield of view potentially facilitates a simulated-environment portrayalof the user-interaction controller and/or the user's body partassociated with the user-interaction controller within the user's fieldof vision. In particular, a pose of the user-interaction controllerand/or the user's body part associated with the user-interactioncontroller in the simulated environment can be portrayed to closelyemulate a pose of the user-interaction controller and/or the user's bodypart associated with the user-interaction controller in the user's fieldof vision in the real-world environment. This would provide the userwith a feeling of immersion within the simulated environment and wouldconsiderably enhance the user's viewing experience.

Throughout the present disclosure, the term “image renderer” refers toequipment that, when operated, renders images of the visual scene.

Optionally, the at least one image renderer is implemented as at leastone display. Optionally, the at least one display is selected from thegroup consisting of: a Liquid Crystal Display (LCD), a Light EmittingDiode (LED)-based display, a micro LED-based display, an Organic LED(OLED)-based display, a micro OLED-based display, a Liquid Crystal onSilicon (LCoS)-based display, a pinhole aperture array-based display,and a Cathode Ray Tube (CRT)-based display.

Optionally, the at least one image renderer is implemented as at leastone projector. Optionally, in this regard, the images are projected ontoa projection screen or directly onto a retina of the user's eyes.Optionally, the at least one projector is selected from the groupconsisting of: an LCD-based projector, an LED-based projector, anOLED-based projector, an LCoS-based projector, a Digital LightProcessing (DLP)-based projector, and a laser projector.

It will be appreciated that the at least one image renderer can bearranged in various ways within the head-mounted display apparatus. Asan example, the at least one image renderer may face the user's eyes. Asanother example, the at least one image renderer may be arranged in aframe of the head-mounted display apparatus, a projection of the imagesof the visual scene being reflected from at least one optical element tobe directed towards the user's eyes.

Optionally, an absolute pose of the head-mounted display apparatuswithin the real-world environment is determined using camera-basedinside-out tracking. Optionally, in this regard, the head-mounteddisplay apparatus comprises:

-   -   at least one camera configured to capture images of the        real-world environment from a perspective of the head-mounted        display apparatus, and    -   a processing unit coupled to the at least one camera, wherein        the processing unit is configured to determine, based on the        captured images, the absolute pose of the head-mounted display        apparatus within the real-world environment.

Optionally, the at least one camera is arranged on an outer surface ofthe head-mounted display apparatus, said outer surface being inproximity of the first portion. Optionally, in such a case, thedetermined absolute pose of the head-mounted display apparatus withinthe real-world environment would correspond to an absolute pose of thefirst receiver within the real-world environment. The processor utilisessaid absolute pose of the first receiver and the relative position andorientation of the first receiver with respect to the at least oneemitter, to determine an absolute pose of the at least one emitterwithin the real-world environment. Furthermore, the processor utilisessaid absolute pose of the at least one emitter and the relative positionand orientation of the second receiver with respect to the at least oneemitter, to determine an absolute pose of the second receiver within thereal-world environment.

Optionally, the at least one camera is at least one of: a digitalcamera, a RGB-D camera, a Light Detection and Ranging (LiDAR) camera, aTime-of-Flight (ToF) camera, a laser rangefinder, a stereo camera.Moreover, optionally, the at least one camera is implemented as at leastone of: an infrared camera, a visible-light camera, a hyperspectralcamera.

In an embodiment, the at least one emitter is arranged on a thirdportion of the head-mounted display apparatus that is different from thefirst portion and the second portion. Notably, the at least one emitteris arranged on the third portion in a manner that there is minimalmisalignment of the at least one emitter when the head-mounted displayapparatus is in use. In an example, the at least one emitter may bearranged on the third portion using an adhesive, an attachment means(for example, such as screws or fasteners), and the like. In anotherexample, the at least one emitter may be snugly fit into a recess withinthe third portion of the head-mounted display apparatus.

It will be appreciated that the third portion of the head-mounteddisplay apparatus is selected in a manner that arranging the at leastone emitter (which is generally larger and heavier in comparison to thefirst and second receivers) at the third portion would not adverselyaffect usability of the head-mounted display apparatus.

Optionally, the third portion is a part of a head strap of thehead-mounted display apparatus. Optionally, in this regard, the thirdportion is a central portion of the head strap. When the user wears thehead-mounted display apparatus on his/her head, the third portion wouldlie on a back side of the user's head. The head strap need not have adirect immutable connection to the first portion.

It will be appreciated that arranging the at least one emitter in thehead strap of the head-mounted display apparatus allows for maintaininga required weight balance of the head-mounted display apparatus withoutadversely impacting wearability of the head-mounted display apparatus.Such an arrangement of the at least one emitter optimally utilizes theweight of the at least one emitter to provide an ergonomic design of thehead-mounted display apparatus.

Optionally, when the first portion is in the proximity of the imagerenderer and the at least one emitter is arranged on the head strap, thefirst receiver arranged on the first portion compensates for possiblemisalignment of the at least one emitter. Notably, a position and anorientation of the at least one emitter may be different for differentusers. As an example, the different users may have different head sizes.A distance between the first portion and the third portion wouldincrease with an increase in a radius of the user's head. As anotherexample, the different users may wear the head-mounted display apparatusdifferently based on at least one of: a hair style of the user, a shapeof the user's head, a comfort of the user. An angle between the firstportion of the head-mounted display apparatus and the third portionwould vary based upon how the user wears the head-mounted displayapparatus. In order to compensate for the aforesaid possiblemisalignment issues, the first receiver is arranged in a known positionand orientation with respect to the image renderer, and the relativeposition and orientation of the first receiver with respect to the atleast one emitter is calculated.

Optionally, the at least one emitter comprises a plurality of emitters,and wherein a given receiver, in operation, senses signals emitted bythe plurality of emitters. In such a case, the processor is configuredto process the generated sensor data to determine relative positions andorientations of the given receiver with respect to each of the pluralityof emitters, and to determine, based on said relative positions andorientations, a relative position and orientation of the given receiverwith respect to another receiver. It will be appreciated that utilizingthe relative positions and orientations of the given receiver withrespect to multiple emitters allows for increasing the accuracy of thetracking system. For example, an average value of said relativepositions and orientations can be used to determine an accurate value ofa relative pose of the given receiver with respect to the plurality ofemitters.

In another embodiment, the at least one emitter is arranged on anexternal device. In such a case, the external device is separate fromthe head-mounted display apparatus. The external device may or may notbe coupled in communication with the head-mounted display apparatus.

Optionally, the external device is a processing device that isconfigured to perform various image processing operations to generatethe images to be rendered via the head-mounted display apparatus.Optionally, in this regard, the external device is communicably coupledto the head-mounted display apparatus wirelessly, or via wires (namely,cables). As an example, the external device may be a virtual realitygame console that generates images for a virtual reality shooting game.

Alternatively, optionally, the external device is a base emitter stationarranged within a given premises, the base emitter station being sharedbetween a plurality of tracking systems. The base emitter station may ormay not be communicably coupled to the head-mounted display apparatus.Moreover, in such a case, a single processor is configured to determinerelative positions and orientations of a plurality of receivers of theplurality of tracking systems with respect to the base emitter station.Moreover, in such a case, a single three-dimensional coordinate systemis employed for tracking poses of a plurality of head-mounted displayapparatuses associated with the plurality of tracking systems. Using thebase emitter station as an origin of said three-dimensional coordinatesystem allows for determining absolute poses of the plurality ofhead-mounted display apparatuses within the given premises withoutimplementing camera-based inside-out tracking.

Optionally, in this regard, a same frequency spectrum is shared betweenthe plurality of head-mounted display apparatuses for tracking the posesof the plurality of head-mounted display apparatuses using the pluralityof tracking systems. In particular, frequency division multiplexing ortime division multiplexing is employed to enable said frequency spectrumsharing. A typical tracking system for a given head-mounted displayapparatus employs two receivers, wherein each of the two receiversrequire one communication channel for their operation. As a result, anumber of head-mounted display apparatuses that can be used in a singlepremises becomes very limited. It will be appreciated that by suchfrequency spectrum sharing, a number of communication channels requiredper head-mounted display apparatus can be reduced, thereby allowing fora larger number of head-mounted display apparatuses to be used andtracked within the single premises.

In an example, different frequencies may be used to drive differentelectromagnetic fields for different tracking systems. For example,within a given premises, a 20 kilohertz frequency may be used to drivean electromagnetic field for a first tracking system and a 25 kilohertzfrequency may be used to drive an electromagnetic field for secondtracking system. Furthermore, time division multiplexing may be employedto utilize a same frequency of electromagnetic signals to drive multipleelectromagnetic fields of multiple tracking systems.

It will be appreciated that within the given premises, the base emitterstation would be arranged in a manner that the emitted signals propagatethrough a considerable region of the given premises. Moreover, thesignals emitted from the base emitter station propagate though a largerregion of the given premises than signals emitted from any singleemitter of a single tracking system. As an example, the base emitterstation may be arranged at a centre of a given exhibition premises toemit signals within a 10000 cubic meters region of said exhibitionpremises.

Optionally, the at least one emitter is arranged at a referencelocation. In such a case, the absolute pose of the at least one emitterwithin the real-world environment is known. Therefore, the processorutilises said absolute pose of the at least one emitter, and thedetermined relative positions and orientations of the first receiver andthe second receiver with respect to the at least one emitter, todetermine the absolute poses of the first receiver and the secondreceiver within the real-world environment.

As an example, four emitters may be arranged at four reference locationswithin a given indoor environment in a manner that signals emitted bythe four emitters propagate through the entire three-dimensional spaceof the given indoor environment. In such an example, the four emittersmay be directly arranged within the given indoor environment or may bearranged on external devices that are placed within the given indoorenvironment.

Optionally, the tracking system further comprises a third receiverarranged at a reference location, wherein the processor is configured todetermine a relative position and orientation of the third receiver withrespect to the at least one emitter, and to determine, based on saidrelative position and orientation, relative positions and orientationsof the first receiver and the second receiver with respect to the thirdreceiver. Optionally, in this regard, the reference location of thethird receiver acts as an origin of a three-dimensional coordinatesystem. It will be appreciated that using the third receiver in theaforesaid manner enables tracking using the tracking system even insituations where certain tracking technologies cannot be implemented(for example, visible-light based tracking is difficult to implement ina dark room).

Optionally, the reference location is a central location of thereal-world environment.

Optionally, the head-mounted display apparatus further comprises atleast one motion sensor, wherein sensor data of the at least one motionsensor is employed for enhancing an accuracy of the tracking system.Optionally, in this case, the at least one motion sensor is at least oneof: an accelerometer, a gyroscope, a magnetometer.

Moreover, optionally, the tracking system employs Kalman filtering oraveraging technique to combine the sensor data of the at least onemotion sensor and the sensor data generated by the first and secondreceivers. Such combined sensor data is then processed by the processorto determine the relative positions and orientations of the firstreceiver and the second receiver with respect to the at least oneemitter, and the relative position and orientation of the secondreceiver with respect to the first receiver. It will be appreciated thatcombining the sensor data enhances an accuracy, stability andreliability of the tracking system.

The present disclosure also relates to the method as described above.Various embodiments and variants disclosed above, with respect to theaforementioned first aspect, apply mutatis mutandis to the method.

Optionally, in the method, the first portion is in a proximity of animage renderer of the head-mounted display apparatus, a relativeposition and orientation of the first receiver with respect to the imagerenderer being known, wherein the method further comprises determining arelative position and orientation of the user-interaction controllerwith respect to a field of view that is visible to the user via theimage renderer, based on the relative position and orientation of thefirst receiver with respect to the image renderer and the relativeposition and orientation of the second receiver with respect to thefirst receiver.

According to an embodiment, in the method, the at least one emitter isarranged on a third portion of the head-mounted display apparatus thatis different from the first portion and the second portion.

Optionally, in the method, the third portion is a part of a head strapof the head-mounted display apparatus.

According to another embodiment, in the method, the at least one emitteris arranged on an external device.

Optionally, in the method, the at least one emitter is arranged at areference location.

Optionally, in the method, the at least one emitter comprises aplurality of emitters, and wherein the step of sensing the emittedsignals comprises sensing, via a given receiver, signals emitted by theplurality of emitters.

Optionally, in the method, a third receiver is arranged at a referencelocation, and the method further comprises

-   -   determining a relative position and orientation of the third        receiver with respect to the at least one emitter; and    -   determining, based on said relative position and orientation,        relative positions and orientations of the first receiver and        the second receiver with respect to the third receiver.

Optionally, in the method, the emitted signals are electromagneticsignals.

Alternatively, optionally, in the method, the emitted signals areacoustic signals.

Optionally, in the method, the generated sensor data comprisesmeasurements of at least one signal characteristic of the signals.

Optionally, in the method, the at least one signal characteristiccomprises at least one of: a strength, a phase, a wavelength, afrequency, a time-of-flight, an angle of arrival, a polarization, aDoppler spread, a delay spread, a delay signature.

DETAILED DESCRIPTION OF THE DRAWINGS

Referring to FIG. 1, illustrated is a block diagram of an architectureof a tracking system 100 for use in a head-mounted display apparatus102, in accordance with an embodiment of the present disclosure. Thetracking system 100 comprises at least one emitter (depicted as anemitter 104), a first receiver 106 and a second receiver 108, and aprocessor 110. The at least one emitter 104, in operation, emitssignals. The first receiver 106 and the second receiver 108, inoperation, sense the emitted signals and generate sensor data. Theprocessor 110 is configured to process the generated sensor data todetermine relative positions and orientations of the first receiver 106and the second receiver 108 with respect to the at least one emitter104, and to determine, based on the determined relative positions andorientations, a relative position and orientation of the second receiver108 with respect to the first receiver 106.

Referring to FIG. 2, illustrated is a block diagram of an architectureof a tracking system 200 for use in a head-mounted display apparatus202, in accordance with another embodiment of the present disclosure.The tracking system 200 comprises at least one emitter (comprising aplurality of emitters 204, 206 and 208), a first receiver 210, a secondreceiver 212, a third receiver 214, and a processor 216. A givenreceiver, in operation, senses signals emitted by the emitters 204, 206and 208.

The third receiver 214 is arranged at a reference location. Theprocessor 216 is configured to determine a relative position andorientation of the third receiver 214 with respect to the emitters 204,206 and 208, and to determine, based on said relative position andorientation, relative positions and orientations of the first receiver210 and the second receiver 212 with respect to the third receiver 214.

It may be understood by a person skilled in the art that FIGS. 1 and 2depict simplified architectures of the tracking systems 100 and 200 forsake of clarity, which should not unduly limit the scope of the claimsherein. The person skilled in the art will recognize many variations,alternatives, and modifications of embodiments of the presentdisclosure.

Referring to FIG. 3, illustrated is a schematic illustration of atracking system 300 in use, in accordance with an embodiment of thepresent disclosure. As shown, the tracking system 300 is used in ahead-mounted display apparatus 302, wherein the head-mounted displayapparatus 302 in operation is worn by the user on his/her head.

The tracking system 300 comprises at least one emitter (depicted as anemitter 304), a first receiver 306 and a second receiver 308, and aprocessor (not shown). The first receiver 306 and the second receiver308 are arranged on a first portion 302A and a second portion 302B ofthe head-mounted display apparatus 302, respectively. The first portion302A faces a user when the head-mounted display apparatus 302 is worn bythe user, and the second portion 302B is a part of a user-interactioncontroller 310 of the head-mounted display apparatus 302.

As shown, the first portion 302A is in a proximity of an image renderer312 of the head-mounted display apparatus 302. Moreover, the at leastone emitter 304 is arranged on a third portion 302C of the head-mounteddisplay apparatus 302 that is different from the first portion 302A andthe second portion 302B. The third portion 302C is a part of a headstrap 314 of the head-mounted display apparatus 302.

The at least one emitter 304, in operation, emits signals. The firstreceiver 306 and the second receiver 308, in operation, sense theemitted signals and generate sensor data. The processor is configured toprocess the generated sensor data to determine relative positions andorientations of the first receiver 306 and the second receiver 308 withrespect to the at least one emitter 304, and to determine, based on thedetermined relative positions and orientations, a relative position andorientation of the second receiver 308 with respect to the firstreceiver 306.

Referring to FIG. 4, illustrated is a schematic illustration of atracking system 400 in use, in accordance with another embodiment of thepresent disclosure. As shown, the tracking system 400 is used in ahead-mounted display apparatus 402, wherein the head-mounted displayapparatus 402 in operation is worn by the user on his/her head.

The tracking system 400 comprises at least one emitter (depicted as anemitter 404), a first receiver 406 and a second receiver 408, and aprocessor (not shown). The first receiver 406 and the second receiver408 are arranged on a first portion 402A and a second portion 402B ofthe head-mounted display apparatus 402, respectively. The first portion402A faces a user when the head-mounted display apparatus 402 is worn bythe user, and the second portion 402B is a part of a user-interactioncontroller 410 of the head-mounted display apparatus 402.

As shown, the first portion 402A is in a proximity of an image renderer412 of the head-mounted display apparatus 402. Moreover, the at leastone emitter 404 is arranged on an external device 414 that iscommunicably coupled to the head-mounted display apparatus 402. Asshown, the external device 414 is communicably coupled to thehead-mounted display apparatus 402 via a cable 416.

The at least one emitter 404, in operation, emits signals. The firstreceiver 406 and the second receiver 408, in operation, sense theemitted signals and generate sensor data. The processor is configured toprocess the generated sensor data to determine relative positions andorientations of the first receiver 406 and the second receiver 408 withrespect to the at least one emitter 404, and to determine, based on thedetermined relative positions and orientations, a relative position andorientation of the second receiver 408 with respect to the firstreceiver 406.

FIGS. 3 and 4 are merely simplified example illustrations of trackingsystems 300 and 400, for sake of clarity only, and should not undulylimit the scope of the claims herein. A person skilled in the art willrecognize many variations, alternatives, and modifications ofembodiments of the present disclosure.

Referring to FIG. 5, illustrated are steps of a method of tracking, inaccordance with an embodiment of the present disclosure. At a step 502,signals are emitted via at least one emitter. At a step 504, the emittedsignals are sensed, and sensor data is generated, via a first receiverand a second receiver. The first receiver and the second receiver arearranged on a first portion and a second portion of a head-mounteddisplay apparatus, respectively. The first portion faces a user when thehead-mounted display apparatus is worn by the user, and the secondportion is a part of a user-interaction controller of the head-mounteddisplay apparatus. At a step 506, the generated sensor data is processedto determine relative positions and orientations of the first receiverand the second receiver with respect to the at least one emitter. At astep 508, a relative position and orientation of the second receiverwith respect to the first receiver is determined, based on thedetermined relative positions and orientations.

The steps 502 to 508 are only illustrative and other alternatives canalso be provided where one or more steps are added, one or more stepsare removed, or one or more steps are provided in a different sequencewithout departing from the scope of the claims herein.

Modifications to embodiments of the present disclosure described in theforegoing are possible without departing from the scope of the presentdisclosure as defined by the accompanying claims. Expressions such as“including”, “comprising”, “incorporating”, “have”, “is” used todescribe and claim the present disclosure are intended to be construedin a non-exclusive manner, namely allowing for items, components orelements not explicitly described also to be present. Reference to thesingular is also to be construed to relate to the plural.

What is claimed is:
 1. A tracking system for use in a head-mounteddisplay apparatus, the tracking system comprising: at least one emitterthat, in operation, emits signals; a first receiver and a secondreceiver that, in operation, sense the emitted signals and generatesensor data, the first receiver and the second receiver being arrangedon a first portion and a second portion of the head-mounted displayapparatus, respectively, wherein the first portion faces a user when thehead-mounted display apparatus is worn by the user, and the secondportion is a part of a user-interaction controller of the head-mounteddisplay apparatus; and a processor configured to process the generatedsensor data to determine relative positions and orientations of thefirst receiver and the second receiver with respect to the at least oneemitter, and to determine, based on the determined relative positionsand orientations, a relative position and orientation of the secondreceiver with respect to the first receiver.
 2. The tracking system ofclaim 1, wherein the first portion is in a proximity of an imagerenderer of the head-mounted display apparatus, a relative position andorientation of the first receiver with respect to the image rendererbeing known, wherein the processor is configured to determine a relativeposition and orientation of the user-interaction controller with respectto a field of view that is visible to the user via the image renderer,based on the relative position and orientation of the first receiverwith respect to the image renderer and the relative position andorientation of the second receiver with respect to the first receiver.3. The tracking system of claim 1, wherein the at least one emitter isarranged on a third portion of the head-mounted display apparatus thatis different from the first portion and the second portion.
 4. Thetracking system of claim 3, wherein the third portion is a part of ahead strap of the head-mounted display apparatus.
 5. The tracking systemof claim 1, wherein the at least one emitter is arranged on an externaldevice.
 6. The tracking system of claim 1, wherein the at least oneemitter is arranged at a reference location.
 7. The tracking system ofclaim 1, wherein the at least one emitter comprises a plurality ofemitters, and wherein a given receiver, in operation, senses signalsemitted by the plurality of emitters.
 8. The tracking system of claim 1,further comprising a third receiver arranged at a reference location,wherein the processor configured to determine a relative position andorientation of the third receiver with respect to the at least oneemitter, and to determine, based on said relative position andorientation, relative positions and orientations of the first receiverand the second receiver with respect to the third receiver.
 9. Thetracking system of claim 1, wherein the emitted signals areelectromagnetic signals.
 10. The tracking system of claim 1, wherein theemitted signals are acoustic signals.
 11. The tracking system of claim1, wherein the generated sensor data comprises measurements of at leastone signal characteristic of the signals.
 12. The tracking system ofclaim 11, wherein the at least one signal characteristic comprises atleast one of: a strength, a phase, a wavelength, a frequency, atime-of-flight, an angle of arrival, a polarization, a Doppler spread, adelay spread, a delay signature.
 13. A method of tracking, the methodcomprising: emitting signals, via at least one emitter; sensing, via afirst receiver and a second receiver, the emitted signals and generatingsensor data, the first receiver and the second receiver being arrangedon a first portion and a second portion of a head-mounted displayapparatus, respectively, wherein the first portion faces a user when thehead-mounted display apparatus is worn by the user, and the secondportion is a part of a user-interaction controller of the head-mounteddisplay apparatus; processing the generated sensor data to determinerelative positions and orientations of the first receiver and the secondreceiver with respect to the at least one emitter; and determining,based on the determined relative positions and orientations, a relativeposition and orientation of the second receiver with respect to thefirst receiver.
 14. The method of claim 13, wherein the first portion isin a proximity of an image renderer of the head-mounted displayapparatus, a relative position and orientation of the first receiverwith respect to the image renderer being known, wherein the methodfurther comprises determining a relative position and orientation of theuser-interaction controller with respect to a field of view that isvisible to the user via the image renderer, based on the relativeposition and orientation of the first receiver with respect to the imagerenderer and the relative position and orientation of the secondreceiver with respect to the first receiver.
 15. The method of claim 13,wherein the at least one emitter is arranged on a third portion of thehead-mounted display apparatus that is different from the first portionand the second portion.
 16. The method of claim 15, wherein the thirdportion is a part of a head strap of the head-mounted display apparatus.17. The method of claim 13, wherein the at least one emitter is arrangedon an external device.
 18. The method of claim 13, wherein the at leastone emitter is arranged at a reference location.
 19. The method of claim13, wherein the at least one emitter comprises a plurality of emitters,and wherein the step of sensing the emitted signals comprises sensing,via a given receiver, signals emitted by the plurality of emitters. 20.The method of claim 13, wherein a third receiver is arranged at areference location, and wherein the method further comprises determininga relative position and orientation of the third receiver with respectto the at least one emitter; and determining, based on said relativeposition and orientation, relative positions and orientations of thefirst receiver and the second receiver with respect to the thirdreceiver.
 21. The method of claim 13, wherein the emitted signals areelectromagnetic signals.
 22. The method of claim 13, wherein the emittedsignals are acoustic signals.
 23. The method of claim 13, wherein thegenerated sensor data comprises measurements of at least one signalcharacteristic of the signals.
 24. The method of claim 23, wherein theat least one signal characteristic comprises at least one of: astrength, a phase, a wavelength, a frequency, a time-of-flight, an angleof arrival, a polarization, a Doppler spread, a delay spread, a delaysignature.